Statistical Mechanics of Learning and Optimization
نویسنده
چکیده
منابع مشابه
Product Distributions for Distributed Optimization
With connections to bounded rational game theory, information theory and statistical mechanics, Product Distribution (PD) theory provides a new framework for performing distributed optimization. Furthermore, PD theory extends and formalizes Collective Intelligence, thus connectingt distributed optimization to distributed Reinforcement Learning (RL). This paper provides an overview of PD theory ...
متن کاملOptimization of the Asymptotic Property of Mutual Learning Involving an Integration Mechanism of Ensemble Learning
Abstruct– We propose an optimization method of mutual learning which converges into the identical state of optimum ensemble learning within the framework of on-line learning, and have analyzed its asymptotic property through the statistical mechanics method.The proposed model consists of two learning steps: two students independently learn from a teacher, and then the students learn from each o...
متن کاملStatistical Mechanics of Unsupervised Structure Recognition
A model of unsupervised learning is studied, where the environment provides N{ dimensional input examples that are drawn from two overlapping Gaussian clouds. We consider the optimization of two diierent objective functions: the search for the direction of the largest variance in the data and the largest separating gap (stability) between clusters of examples respectively. By means of a statist...
متن کاملPhase Transitions in Semidefinite Relaxations
Statistical inference problems arising within signal processing, data mining, and machine learning naturally give rise to hard combinatorial optimization problems. These problems become intractable when the dimensionality of the data is large, as is often the case for modern datasets. A popular idea is to construct convex relaxations of these combinatorial problems, which can be solved efficien...
متن کاملOptimisation of on–line principal component analysis
Various techniques, used to optimise on-line principal component analysis, are investigated by methods of statistical mechanics. These include local and global optimisation of node-dependent learning-rates which are shown to be very efficient in speeding up the learning process. They are investigated further for gaining insight into the learning rates’ timedependence, which is then employed for...
متن کامل